Skip to content

[SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter#37904

Closed
HyukjinKwon wants to merge 2 commits intoapache:masterfrom
HyukjinKwon:fix-linter
Closed

[SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter#37904
HyukjinKwon wants to merge 2 commits intoapache:masterfrom
HyukjinKwon:fix-linter

Conversation

@HyukjinKwon
Copy link
Copy Markdown
Member

@HyukjinKwon HyukjinKwon commented Sep 16, 2022 •
edited
Loading

What changes were proposed in this pull request?

This PR sets the upperbound for pyzmq as <24.0.0 in our CI Python linter job. The new release seems having a problem (zeromq/pyzmq@2d3327d).

Why are the changes needed?

To fix the linter build failure. See https://github.com/apache/spark/actions/runs/3063515551/jobs/4947782771

      /tmp/timer_created_0ftep6.c: In function ‘main’:
      /tmp/timer_created_0ftep6.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
          2 |     timer_create();
            |     ^~~~~~~~~~~~
      x86_64-linux-gnu-gcc -pthread tmp/timer_created_0ftep6.o -L/usr/lib/x86_64-linux-gnu -o a.out
      /usr/bin/ld: tmp/timer_created_0ftep6.o: in function `main':
      /tmp/timer_created_0ftep6.c:2: undefined reference to `timer_create'
      collect2: error: ld returned 1 exit status
      no timer_create, linking librt
      ************************************************
      building 'zmq.libzmq' extension
      creating build/temp.linux-x86_64-cpython-39/buildutils
      creating build/temp.linux-x86_64-cpython-39/bundled
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq/src
      x86_64-linux-gnu-g++ -pthread -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/include/python3.9 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-39/buildutils/initlibzmq.o
      buildutils/initlibzmq.cpp:10:10: fatal error: Python.h: No such file or directory
         10 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects

Does this PR introduce any user-facing change?

No, test-only.

How was this patch tested?

CI in this PRs should validate it.

@github-actions github-actions bot added the INFRA label Sep 16, 2022
@HyukjinKwon HyukjinKwon changed the title [SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for linters [SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter Sep 16, 2022
dongjoon-hyun
dongjoon-hyun approved these changes Sep 16, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @HyukjinKwon !

HyukjinKwon added a commit that referenced this pull request Sep 16, 2022
@HyukjinKwon
[SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter
507708d 
### What changes were proposed in this pull request?

This PR sets the upperbound for `pyzmq` as `<24.0.0` in our CI Python linter job. The new release seems having a problem (zeromq/pyzmq@2d3327d).

### Why are the changes needed?

To fix the linter build failure. See https://github.com/apache/spark/actions/runs/3063515551/jobs/4947782771

```
      /tmp/timer_created_0ftep6.c: In function ‘main’:
      /tmp/timer_created_0ftep6.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
          2 |     timer_create();
            |     ^~~~~~~~~~~~
      x86_64-linux-gnu-gcc -pthread tmp/timer_created_0ftep6.o -L/usr/lib/x86_64-linux-gnu -o a.out
      /usr/bin/ld: tmp/timer_created_0ftep6.o: in function `main':
      /tmp/timer_created_0ftep6.c:2: undefined reference to `timer_create'
      collect2: error: ld returned 1 exit status
      no timer_create, linking librt
      ************************************************
      building 'zmq.libzmq' extension
      creating build/temp.linux-x86_64-cpython-39/buildutils
      creating build/temp.linux-x86_64-cpython-39/bundled
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq/src
      x86_64-linux-gnu-g++ -pthread -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/include/python3.9 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-39/buildutils/initlibzmq.o
      buildutils/initlibzmq.cpp:10:10: fatal error: Python.h: No such file or directory
         10 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
```

### Does this PR introduce _any_ user-facing change?

No, test-only.

### How was this patch tested?

CI in this PRs should validate it.

Closes #37904 from HyukjinKwon/fix-linter.

Lead-authored-by: Hyukjin Kwon <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 254bd80)
Signed-off-by: Hyukjin Kwon <[email protected]>
HyukjinKwon added a commit that referenced this pull request Sep 16, 2022
@HyukjinKwon
[SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter
8068bd3 
This PR sets the upperbound for `pyzmq` as `<24.0.0` in our CI Python linter job. The new release seems having a problem (zeromq/pyzmq@2d3327d).

To fix the linter build failure. See https://github.com/apache/spark/actions/runs/3063515551/jobs/4947782771

```
      /tmp/timer_created_0ftep6.c: In function ‘main’:
      /tmp/timer_created_0ftep6.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
          2 |     timer_create();
            |     ^~~~~~~~~~~~
      x86_64-linux-gnu-gcc -pthread tmp/timer_created_0ftep6.o -L/usr/lib/x86_64-linux-gnu -o a.out
      /usr/bin/ld: tmp/timer_created_0ftep6.o: in function `main':
      /tmp/timer_created_0ftep6.c:2: undefined reference to `timer_create'
      collect2: error: ld returned 1 exit status
      no timer_create, linking librt
      ************************************************
      building 'zmq.libzmq' extension
      creating build/temp.linux-x86_64-cpython-39/buildutils
      creating build/temp.linux-x86_64-cpython-39/bundled
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq/src
      x86_64-linux-gnu-g++ -pthread -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/include/python3.9 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-39/buildutils/initlibzmq.o
      buildutils/initlibzmq.cpp:10:10: fatal error: Python.h: No such file or directory
         10 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
```

No, test-only.

CI in this PRs should validate it.

Closes #37904 from HyukjinKwon/fix-linter.

Lead-authored-by: Hyukjin Kwon <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 254bd80)
Signed-off-by: Hyukjin Kwon <[email protected]>
HyukjinKwon added a commit that referenced this pull request Sep 16, 2022
@HyukjinKwon
[SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter
bc9e434 
This PR sets the upperbound for `pyzmq` as `<24.0.0` in our CI Python linter job. The new release seems having a problem (zeromq/pyzmq@2d3327d).

To fix the linter build failure. See https://github.com/apache/spark/actions/runs/3063515551/jobs/4947782771

```
      /tmp/timer_created_0ftep6.c: In function ‘main’:
      /tmp/timer_created_0ftep6.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
          2 |     timer_create();
            |     ^~~~~~~~~~~~
      x86_64-linux-gnu-gcc -pthread tmp/timer_created_0ftep6.o -L/usr/lib/x86_64-linux-gnu -o a.out
      /usr/bin/ld: tmp/timer_created_0ftep6.o: in function `main':
      /tmp/timer_created_0ftep6.c:2: undefined reference to `timer_create'
      collect2: error: ld returned 1 exit status
      no timer_create, linking librt
      ************************************************
      building 'zmq.libzmq' extension
      creating build/temp.linux-x86_64-cpython-39/buildutils
      creating build/temp.linux-x86_64-cpython-39/bundled
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq/src
      x86_64-linux-gnu-g++ -pthread -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/include/python3.9 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-39/buildutils/initlibzmq.o
      buildutils/initlibzmq.cpp:10:10: fatal error: Python.h: No such file or directory
         10 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
```

No, test-only.

CI in this PRs should validate it.

Closes #37904 from HyukjinKwon/fix-linter.

Lead-authored-by: Hyukjin Kwon <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 254bd80)
Signed-off-by: Hyukjin Kwon <[email protected]>
@HyukjinKwon
Copy link
Copy Markdown
Member Author

HyukjinKwon commented Sep 16, 2022

Merged to master, branch-3.3, branch-3.2, and branch-3.1.

@Yikun Yikun mentioned this pull request Sep 16, 2022
Closed
LuciferYang pushed a commit to LuciferYang/spark that referenced this pull request Sep 20, 2022
@HyukjinKwon @LuciferYang
[SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter
2e0a528 
### What changes were proposed in this pull request?

This PR sets the upperbound for `pyzmq` as `<24.0.0` in our CI Python linter job. The new release seems having a problem (zeromq/pyzmq@2d3327d).

### Why are the changes needed?

To fix the linter build failure. See https://github.com/apache/spark/actions/runs/3063515551/jobs/4947782771

```
      /tmp/timer_created_0ftep6.c: In function ‘main’:
      /tmp/timer_created_0ftep6.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
          2 |     timer_create();
            |     ^~~~~~~~~~~~
      x86_64-linux-gnu-gcc -pthread tmp/timer_created_0ftep6.o -L/usr/lib/x86_64-linux-gnu -o a.out
      /usr/bin/ld: tmp/timer_created_0ftep6.o: in function `main':
      /tmp/timer_created_0ftep6.c:2: undefined reference to `timer_create'
      collect2: error: ld returned 1 exit status
      no timer_create, linking librt
      ************************************************
      building 'zmq.libzmq' extension
      creating build/temp.linux-x86_64-cpython-39/buildutils
      creating build/temp.linux-x86_64-cpython-39/bundled
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq/src
      x86_64-linux-gnu-g++ -pthread -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/include/python3.9 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-39/buildutils/initlibzmq.o
      buildutils/initlibzmq.cpp:10:10: fatal error: Python.h: No such file or directory
         10 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
```

### Does this PR introduce _any_ user-facing change?

No, test-only.

### How was this patch tested?

CI in this PRs should validate it.

Closes apache#37904 from HyukjinKwon/fix-linter.

Lead-authored-by: Hyukjin Kwon <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
sunchao pushed a commit to sunchao/spark that referenced this pull request Jun 2, 2023
@HyukjinKwon @sunchao
[SPARK-40461][INFRA] Set upperbound for pyzmq 24.0.0 for Python linter
0b6b2f2 
This PR sets the upperbound for `pyzmq` as `<24.0.0` in our CI Python linter job. The new release seems having a problem (zeromq/pyzmq@2d3327d).

To fix the linter build failure. See https://github.com/apache/spark/actions/runs/3063515551/jobs/4947782771

```
      /tmp/timer_created_0ftep6.c: In function ‘main’:
      /tmp/timer_created_0ftep6.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
          2 |     timer_create();
            |     ^~~~~~~~~~~~
      x86_64-linux-gnu-gcc -pthread tmp/timer_created_0ftep6.o -L/usr/lib/x86_64-linux-gnu -o a.out
      /usr/bin/ld: tmp/timer_created_0ftep6.o: in function `main':
      /tmp/timer_created_0ftep6.c:2: undefined reference to `timer_create'
      collect2: error: ld returned 1 exit status
      no timer_create, linking librt
      ************************************************
      building 'zmq.libzmq' extension
      creating build/temp.linux-x86_64-cpython-39/buildutils
      creating build/temp.linux-x86_64-cpython-39/bundled
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq
      creating build/temp.linux-x86_64-cpython-39/bundled/zeromq/src
      x86_64-linux-gnu-g++ -pthread -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/include/python3.9 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-39/buildutils/initlibzmq.o
      buildutils/initlibzmq.cpp:10:10: fatal error: Python.h: No such file or directory
         10 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
```

No, test-only.

CI in this PRs should validate it.

Closes apache#37904 from HyukjinKwon/fix-linter.

Lead-authored-by: Hyukjin Kwon <[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 254bd80)
Signed-off-by: Hyukjin Kwon <[email protected]>
@HyukjinKwon HyukjinKwon deleted the fix-linter branch January 15, 2024 00:49
@zhengruifeng zhengruifeng mentioned this pull request Feb 13, 2026
Closed
HyukjinKwon pushed a commit that referenced this pull request Feb 19, 2026
@zhengruifeng @HyukjinKwon
[SPARK-55518][INFRA][PYTHON][DOCS] Upgrade Python to 3.12 in doc build
eb16738 
### What changes were proposed in this pull request?
Upgrade Python to 3.12 in doc build

### Why are the changes needed?
1, Upgrade Python to 3.12 in doc build;
2, Unpin `pyzmq<24.0.0` introduced in #37904 for python linter, otherwise the python installation fails

### Does this PR introduce _any_ user-facing change?
No, infra-only

### How was this patch tested?
CI, the PR builder should cover this change

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #54310 from zhengruifeng/doc_py312.

Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees

No one assigned

Labels

INFRA

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@HyukjinKwon @dongjoon-hyun
⚡